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* |s your friend — albeit a critical friend 

* Helps you work out how to make your project fit for purpose 
and participants 

* |s part of planning a project, not something you think about at 
the end 

e Should be used strategically 

* Helps you work through the iterative cycle that leads to quality 
engagement 
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NPC's four pillar approach 


Effective measurement 
framework developed 


Map your 
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Logic models 


INPUTS D ACTIVITIES OUTCOMES 


© NCCPE 2022 | publicengagement.ac.uk | @NCCPE 


PURPOSE 


PARTICIPANTS 


INPUTS ACTIVITIES OUTPUTS OUTCOMES IMPACTS 


ASSUMPTIONS FACTORS/ RISKS 


EVALUATION 


PURPOSE: To raise awareness of the importance of pollen producing plants for bees 


PARTICIPANTS: Gardeners; local garden centre 


INPUTS ACTIVITIES OUTPUTS OUTCOMES IMPACTS 
e Researcher time, “ Stand at local * 400 people * Improved knowledge of * Participants change 
knowledge and garden centre engaged participants and garden behaviour 
expertise * Meet the * Materials centre staff * Garden centre reduces 
e Funding - £500 researcher e 400 seed packets — * Researcher stocking non pollen 
sessions given away understanding producing plants. 

* Seed giveaway gardeners and garden œ Researchers develop 
centre motivations * more effective 
relating to non pollen interventions 
producing plants * Improved bee habitats 

ASSUMPTIONS FACTORS/ RISKS 
* Gardeners will be willing to participate in the discussion/ * Garden centre opposes the messaging due to 
change behaviours conflict with stocking 


* Participants will plant the seeds 


EVALUATION 


NPC's four pillar approach 


Map your 
theory of 
change 
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Types of Evaluation 


Formative 


Front-end Summative 


Your project 


» [Ime 
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What do you want to know 


Descriptive questions Process questions 
What has happened or how things are Whether an intervention can be considered a success, 
* What were the resources used by the programme an improvement or the best option 
directly and indirectly? * In what ways and for whom was the programme 
* What activities occurred? successful? 
* What changes were observed in conditions or in the * Did the programme provide value for money, taking 
participants? into the account all the costs incurred (not only the 
direct funding) and any negative outcomes? 


Causal questions Action questions 

What has contributed to changes that have been What should be done to respond to evaluation findings 

observed * What changes should be made to address problems 

* What produced the outcomes and impacts? that have been identified? 

* What was the contribution of the programme to * What should be retained or added to reinforce existing 
producing the changes that were observed? strengths? 

* What other factors or programmes contributed to the * Should the programme be refunded? 
observed changes? 


Source: Better Evaluation. (2016) Specify the Key Evaluation Questions (KEQs) 
https://www.betterevaluation.org/en/start here/which method or process 
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Mapping questions to the logic model 


PURPOSE: To raise awareness of the importance of pollen producing plants for bees 


PARTICIPANTS: Gardeners; local garden centre 


INPUTS ACTIVITIES OUTPUTS OUTCOMES IMPACTS 

* Researcher * Standatlocal + 400 people e Improved knowledge œ Participants change 
time, garden centre engaged of participants and behaviour 
knowledge and * Meet the * Materials garden centre staff * Garden centre reduces 
expertise researcher e 400 seed packets * Researcher stocking non pollen 

* Funding - £500 sessions given away understanding producing plants. 

e Seed giveaway gardeners and garden * Researchers develop 
centre motivations * more effective 
relating to non pollen interventions 
producing plants * Improved bee habitats 

ASSUMPTIONS FACTORS/ RISKS 

e Gardeners will be willing to participate in the discussion/ + Garden centre opposes the messaging due to 
change behaviours conflict with stocking 

* Participants will plant the seeds 

EVALUATION 
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NPC's four pillar approach 
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Choose your level of evidence 


Quantitative / 
Qualitative / 


Mixed methods 
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NPC's four pillar approach 


Map your 
theory of 
change 


© NCCPE 2022 | 


Prioritise 
what you 
measure 


Choose Select 
your your 
level of sources 
evidence and tools 


Strategic vision 
Leadership 
Case for impact measurement 


publicengagement.ac.uk | @NCCPE 


Example sources and tools 
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~ Graffiti wall 


© NCCPE 2022 | publicengagement.ac.uk | @NCCPE 


Data collection decisions 


Which techniques are right 
in what circumstances? 
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Data collection 


* action plans or new 
year resolutions 


* activity-based focus 
groups 
* adverts 


* dropping pebbles in 
boxes or stickers on 
charts 


eemotive cartoons 
* focus groups 
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* graffiti wall 

e interviews 

e live Twitter feedback 

* observations 

* opinion postcards 

* peer interviews 

* personal meaning 
mapping 

* photograph diary 


* physical ranking scales 
using people 

* predict 

* questionnaires 

* quirky questionnaires 

ethe string 

“thought boxes 
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* Suitability for the participants in 
the evaluation 


* Ability to analyse the resulting 
data 


e Ethical considerations 
e Sampling 
* Coding 
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The GLO framework 


Information, facts, making 
sense of something, 
adding to prior 


KNOWLEDGE & - knowledge, making 
UNDERSTANDING connections 


Feelings, changes in attitude 
and perception, empathy, 


increased motivation to do N 
something or try something 
new, pride 


ATTITUDES 
& VALUES 


GENERIC 
LEARNING 


OUTCOMES Finding out how to do 


something, communication, 
specialist, physical, academic, 
ENJOYMENT BEHAVIOUR & 
Oy MENE surprise, fun, INSPIRATION PROGRESSION new skills, development or 
creativity, exploratory . CREATIVITY | enhancement of existing skills 
behaviour, experimentation 


Actions of people (past, present and 
future intentions), changes in thoughts 


NPC's four pillar approach 


Map your 
theory of 
change 
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What goes in an evaluation plan? 


. Aim (what do you want to achieve? Big picture!) 

. Objectives (what you need to do to achieve your aim?) 

. Outputs, outcomes and impacts (what will happen as a result?) 
. Evaluation questions (what do you want to know?) 

. Methodology (what strategy will you use?) 


ou Bu N m 


. Data collection (what techniques will you use to collect your 
evidence?) 


6. Data analysis (how will you analyse your data?) 
7. Reporting (who will be reading your report?) 
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Centre for 


National 
| | | Co-ordinating 
| Public Engagement 


The National Co-ordinating Centre for Public Engagement (NCCPE) is 
internationally recognised for its work supporting and inspiring universities 
to engage with the public. 


We work to change perspectives, promote innovation, and nurture and 
celebrate excellence. We also champion meaningful engagement that makes 
a real and valued difference to people's lives. 

The NCCPE is supported by the UK Higher Education Councils, Research 


Councils UK and Wellcome, and has been hosted by the University of Bristol 
and the University of the West of England since it was established in 2008. 
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National Co-ordinating Centre 
for Public Engagement 


2nd Floor, Arnolfini 
16 Narrow Quay 
Bristol, BS1 4QA 


Tel 0117 328 7190 
Email nccpe.enquiries(Quwe.ac.uk 
Twitter @NCCPE 


publicengagement.ac.uk 
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